In automated video surveillance systems, a computer processes video from surveillance cameras to detect “interesting” activity, where that interesting activity can be any motion within the field of view of the camera. In a typical video surveillance system, moving objects are first detected by a detector, e.g., using “background subtraction” (BGS) or some other technique, and are then tracked by a tracker. Object detection utilizes a set of technologies that can detect moving objects in a video sequence generated by a static camera. The detection techniques are tolerant to changes in natural lighting, reasonable changes in the weather, distracting movements (like trees waving in the wind), and camera shake. Object tracking utilizes a set of technologies that can track the shape and position of multiple objects as they move around a space that is monitored by a static camera. Current techniques attempt to handle significant occlusions as objects interact with one another.
For simplicity, the detector and tracker are typically connected in a feed-forward manner with information being passed from the detector to the tracker. In other words, once a moving object is detected by the detector, information about the object (e.g., position, shape, size, color, motion, etc.) can be passed to the tracker, which can utilize the information to track the movement of the object.
Examples of such systems are for example described in US Patent Application No. 2006/0067562 entitled “Detection of moving objects in a video,” filed on Mar. 30, 2006 by Kamath et al.; as well as “Smart Surveillance System” A. Hampapur, L. Brown, J. Connell, S. Pankanti, A. W. Senior, and Y.-L. Tian, Smart Surveillance: Applications, Technologies and Implications, IEEE Pacifc-Rim Conference on Multimedia, Singapore, December 2003, the contents of which are hereby incorporated by reference.
Unfortunately, there are a number of problems in utilizing the aforementioned feed forward arrangement of BGS followed by tracking. One such issue is referred to as “healing,” in which a moving object stops and is incorporated into the background, or a static object moves away and the hole is adapted into the background. In a very simple system, this may happen by a slow blurring of the color of a background pixel towards a new value. When this occurs, the object detector may lose its fix on the object. Accordingly, a need exists for a more robust video surveillance system to eliminate such issues.